Bootstrap Out of Sample Predictive Evaluation Tests∗

We consider the comparison of multiple (possibly all misspecified) models in terms of their out of sample predictive ability. Typically, candidate models compared contain parameters estimated using recursive (or related rolling) estimation schemes. In some cases, predictive evaluation tests have a limiting distribution which is a functional over a Gaussian process, with a covariance kernel that reflects the contribution of parameter uncertainty. The limiting distributions are not nuisance parameter free and valid critical values are thus generally obtained via the bootstrap. Given these considerations, our approach in this paper is to develop a bootstrap procedure that properly captures the contribution of parameter estimation error in recursive estimation schemes. Intuitively, when parameters are estimated recursively, as in done in our framework, earlier observations in the sample are used more heavily than subsequent observations. However, in the standard block bootstrap, all blocks have equal chance of being drawn. This induces a location bias in the bootstrap distribution, which can be either positive or negative across different samples. Within the context of tests of predictive accuracy, we suggest an appropriate recentering of the bootstrap score. The usefulness of our approach is illustrated via two applications: one is an extension of White (2000) reality check to the case of non vanishing parameter estimation error. The oother is an out of sample version of integrated conditional moment tests of Bierens (1982, 1990) and Bierens and Ploberger (1997). The main findings from a small Monte Carlo experiment indicate that the .................... JEL classification: C22, C51.